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Abstract 

In related research on queuing systems, in order to deternnine the system state, there is a widespread practice to assume 
that the system is stable and that distributions of the customer arrival ratio and service ratio are known information. In this 
study, the queuing system is looked at as a black box without any assumptions on the distribution of the arrival and service 
ratios and only keeping the assumption on the stability of the queuing system. By applying the principle of maximum 
entropy, the performance distribution of queuing systems is derived from some easily accessible indexes, such as the 
capacity of the system, the mean number of customers in the system, and the mean utilization of the servers. Some special 
cases are modeled and their performance distributions are derived. Using the chi-square goodness of fit test, the accuracy 
and generality for practical purposes of the principle of maximum entropy approach is demonstrated. 
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introduction 

Queuing theory is mainly regarded as a branch of applied 
probability theory. Its applications are in different fields, such as 
communication networks, computer systems, machine plants, and 
services. Fig. I is a typical queuing system with a single server. 

Queuing theory tries to answer questions like the mean waiting 
time in the queue, the mean system response time (waiting time in 
the queue plus service time), the mean utilization of the service 
facility, the distribution of the number of customers in the queue, 
and the distribution of the number of customers in the system. 
These questions are mainly investigated in a stochastic scenario, 
where, for example, the inter-arrival times of the customers or the 
serving times are assumed to be random typically Poisson arrivals 
and to have exponent distribution serving times. 

However, usually we are mainly interested in steady state 
solutions (see Figure 2); that is, where the system after a long 
running time tends to reach a stable state in which, for example, 
the distribution of customers in the system does not change 
(limiting distribution). 

In a canonical way, the steady state of system performance can 
be derived from assumptions on the distribution of inter-arrival 
times and service times. Hence, these assumptions are the basic 
requirement for analyzing queuing systems. However, in practical 
situations the pre-assumed distributions are difficult to satisfy or to 
acquire. To some degree, this fact limits the practical applications 
of queuing theory. 

The maximum entropy principle is applicable to queuing theory 
because very often only partial information is available about the 
probability distributions. With respect to queuing theory, the 



principle of maximum entropy has been applied to solving 
numerous systems including, but not limited to, M/G/\ and 
M/G/\ queues ([1,2]), finite and infinite capacity G/Gl queues 
([3,4]), multi-server queues ([5,6]), multiple class queues with 
priorities ([7]), and queues with vacation ([8-10]) and queuing 
networks ([11-13]). In fact, since the early 1970's many attempts 
have been made to apply the method of maximum entropy in the 
field of queuing theory. Ferdinand [14] used the method to derive 
the equilibrium solution of the M / M /\/N system by analogy 
with statistical mechanics. Shore [15] built an abstract model from 
which he determined the maximum entropy solution of the 
MjMjcxj and M / M / txj / /N systems. Bard [16] applied entropy 
maximization to a class of problems in the performance evaluation 
of computer systems. El-Affendi and Kouvatsos [2] used the 
maximum entropy principle to analyze the M/G/I— and 
G/M/ 1 -queuing systems at equilibrium. Alfa and Chen [17] 
developed a discrete time approach model for obtaining the 
expected queue length of the M{t)/ G/\ queue. Arizono, Cui, and 
Ohta [18] analyzed M/M/S using the maximum entropy 
principle. Falin, Martin, and Artalejo [19] presented information 
on theoretic approximations for the M/G/\ queue with retrials. 
Kouvatsos and Tabet-Aouel [20] applied entropy maximization to 
characterize the distributional form of the steady-state probabil- 
ities of a G I G I c j PR queue with c(>2) parallel servers and 
R{ > 2) priority classes under a pre-emptive resume (PR) rule. Tadj 
and Hamdi [21] dealt with a quorum queuing system with a 
threshold level r{>\) that regulates the beginning and ending of 
idle and busy periods as follows: an idle period starts when the 
queue size drops below level r and a busy period starts as soon as 
the queue accumulates the same number r. The single server 
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Figure 1. Queuing System Structure with Single-Server. 
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processes r customers in one batcli. Tliey denoted this system as 
M/C/X by Kendall's notation. They presented a maximum 
entropy model to determine the distribution of related random 
variables with known server utilization and mean queue length. 

The work mentioned above applied the principle of maximum 
entropy to certain kinds of queuing systems based on assumptions 
concerning inter-arrivals and server times, which limited the 
practical apphcations of queuing theory. In fact, if a queue system 
is stable, this is not necessary when the maximum entropy method 
is applied to a certain queuing system. In this study, we consider 
the queuing system as a black box and derive a performance index 
for the queuing system by the principle of maximum entropy only 
on the assumption that the queue is stable instead of making 
assumptions on the distribution of inter-arrival times and service 
times. Meanwhile, from the viewpoint of expanding the practical 
application of queuing systems, we use some easily accessible 
indexes of queuing systems, such as the capacity of the system, the 
mean of customers in the queue, and the mean utilization of the 
system. Based on these indexes and the principle of maximum 
entropy, optimization models are then developed to derive the 
performance of queuing systems. 

This paper is organized as follows. Section 2 is a simple review 
of the maximum entropy principle. Sections 3 and 4 develop 
dilferent maximum entropy models with known mean numbers of 
customers and the average value of busy periods under unlimited 
and limited server capacity, respectively. Section 5 compares our 
results with general models with known assumptions on the 
distributions of inter-arrival times and server times by the 
goodness of fit test. The last section concludes. 

Methods: The Principle of Maximum Entropy 

The principle of maximum entropy provides a solution to the 
old problem of the assignment of a probabiUty distribution to a 



random variable that avoids bias while satisfying given or known 
information about the random variable. Jaynes is credited with 
having formalized the principle of maximum entropy in [22]. 

Mathematically the principle can be presented as follows: 
consider a system 0 that has a finite or countable infinite set 6 of 
possible states dQ,6i, - ■ ■ ,d,„ - ■ ■. het p(6i)=pj be the probability 
that the system 0 is in state 9,. Suppose all that is known about 
these probabilities are (m+l) constraints of the form 

& = 1, Pi>0 (1) 



n 

^fk(0)Pi = Mk, i<k<m<ca (2) 

; = 0 

where {M/^} are expectations defined on a set of suitable functions 
\fk{d)}, which can be looked at as the known information. Since, 
in general, the number of constraints is less than the number of 
possible states, one is faced with an infinite number of distributions 
{pi} that satisfy these constraints. The problem is which one to 
choose. 

The maximum entropy principle states that, of all the 
distributions satisfying the constraints supphed by the given 
information, the minimally prejudiced distribution that should 
be chosen is the one that maximizes the system entropy. 



mp)=-j2pi'^^Pi- (3) 

/ = 0 
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Figure 2. Stable and Unstable Queueing System. 
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To sum up, the maximum entropy principle can be described as 
the following model for discrete variants. 



n 

max H(p)= — y^pjlnpi 

1 = 0 

P" (4) 
s.t. < " 

' ']^fk(e)pi = Mk, \<k<m<co 
./)i>0, i = 0,l,2,---,« 

This is a natural extension of Laplace's famous principle of 
insufficient reason, which postulates that the uniform distribution 
is the most satisfactory representation of our knowledge when we 

know nothing about the random variable except that each 
probability is non-negative and that the sum of the probabilities 
is unity. 

The maximization of (3) subject to constraints (1) and (2) can be 
solved by using the Lagrangian method of undetermined 
multipliers leading to the solution: 



Pi = exp ( - A,o - Xxfi (9) Kf„m, / = 0, 1 ,2, • • • ,n (5) 

where {'^k} are the Lagrangian multipliers corresponding to the 
set of constraints (1) and (2). More details on the maximum 
entropy principle and its generalizations can be found in [23,24]. 

Especially, if only the expected value E is known, the second 
constraint converts to 



n 

. 1 = 0 

Hence, 



i^P' =E^al]'^f^ a - EW = 0 

1 = 0 1 = 0 i = 0 

where the coefficients of fS in the above equation are increasing, 
and (0 — £')<0 and there exists an n that makes (w — ii)>0, so 
that there is only one change in sign in the coefficients of ;6. 
According to Descartes's rule of signs, there is only one positive 
real root to the above equation. This indicates the solution 
uniqueness of the maximum entropy estimation problem even if 
there are only expected value and unit and non-negative 
requirements, which provides the basis for the later study in this 
article. 

The maximum entropy approach to queuing systems is based 
on finding a maximum-entropy performance distribution based on 
the knowledge of some moments of the distribution concerned. To 

simplify, we will only discuss a queuing system with a single ser\'er 
and infinite customers, and where the dispatching rule is FIFO 
(First In First Out). The queuing systems are classified by server 
capacity into two types: queuing systems with either unUmited or 
limited server capacity. In each type, we will estimate the 
distribution of the .system state by the maximum entropy principle 
from the mean number of customers in the system and the average 
value of a busy period. 



and according to (5) the estimated distribution is 



Pi= exp(-Xo-)^iO, / = 0,1,2, 



To be simple, let 



a=exp(-Afl), j8=exp(-X,i) 



Discussion 

1. Queuing system with unlimited server capacity 

If it has unlimited server capacity, a cjuc'uing system can serve as 
many customers as possible. Generally speaking, it is hard to 
discover the distributions of customer arrivals and departures. 
Hence, for a queuing system it is easy and practical to acquire 
knowledge of the mean number of customers and the busy periods 
etc. if the queue is stable. 

Let the mean number of customers in the system under steady 
state be Z.,, (>0), and />,- is the probability of the fact that there are 
((/ = 0,1,2,3, • • • ,) customers in the queuing system. According to 
the maximum entropy principle, the following model can be 
established if there is no more information. 



then 



Pi = ixP', i'=l,2, 



Substituting the above equation into the constraints, we can get 



00 

max S= — y^pilnpi 

1 = 0 

/ CO 



S.t.< 



(a) 



^i>,=Lj, (b) 



i=0 



I Pi>0, ! = 0,1,2,---, 



(6) 
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Based on the method in Section 2, the distribution of the system 
state is 



j7,. = e-(i+^o)-M' = e-(i+V(e-My : =ab' (7) 
with (6. a) and (6.b), 



i = 0 

CO 

iab'=Ls 



(8) 



As we know, the above equations have only one positive real root. 
As for y^^i, b' = T^'-r < 1), we can get 



that the maximum entropy distribution with a known non- 
negative mean value is a Poisson distribution [25] . 

Another situation is that the queuing system has unlimited server 

capacity and a known mean server utilization of of 1 —po. Under 
this situation, the maximum entropy model is changed to be: 



max S = — Pi In Pi 

i = 0 

^Pi = 1 -Po 



s.t.< 



;=1 

00 

1 = 0 

I Pi>0,i = 0,l,2,---, 
With a similar approach, we can get 



(13) 



so with (8) 



^ b 
l-b (l-bf 



then, 



1 



Pi = ab'-- 



l+L, \ l+L. 



Let 



then 



Pi = i^-P)p' 



(9) 



(10) 



(11) 



(12) 



The results in (12) reflect the probability distribution function of 
a single server queuing system with a known mean number of 
customers and without limitation on system capacity, from which 
we can obtain the performance indexes of this kind of queuing 
system. It should be noted that the results presented here are 
coherent with a M/M/l queuing system. The intrinsic reason is 



where 



that makes 



Pi = ab' 



ab , ab ^ 

{\-bf 



l-b 



«=7 — r, — ' 1"*=^^ — 



(14) 



(15) 



(16) 



then 



Pi=- 



{\-pof /L,-l+po\ 



-l+PO 



i=l,2,- 



(17) 



Then, with a known Lg and po, the system performance can be 

achieved. 

2. Queuing system with limited server capacity 

In this section we will study the situation where the queuing 
system has a limited capacity; that is, there are customers at 
most in the system, A^ + 1 customers will leave, and the other 
assumptions are the same as before. 

Firsdy, we will consider the queuing system with only a known 
mean customer Z,j. Then, the maximum entropy model is: 



max S 



Y^Pi ln;?i 

i = 0 



s.t.< 



i = 0 

N 



(a) 
(b) 



(18) 



1 = 0 

By a similar method, we come to 
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Pi = ab', / = 0,1, 2, ••■,7V (19) 
where a and b are the roots of the following equations 



/ = 0 

N 

a ib' = Ls 



(20) 



As we know, the above equations have only one positive root. And 
we define 



(21) 



And because 



/(0) = 0,/(l) = A^ 



N+l 



LA,f((X))>0 



(22) 



we can come to the following results: 
N 

i) if -Lj< —, then b<\, 

N , , 

ii) if Ls = —, then 0=1 and the distribution of steady state is an 
average distribution, 

N 

iii) if Ls > y, then b>l. 

Table 1 and Table 2 demonstrate the above results. In Table 1 
the known mean number of customers is assumed to be 6 and in 
Table 2 the number is assumed to be 8. In Table 1, the capacity of 
the system is set to be = 6, when L^ is set to be different values, 
we can get a' and b'vahxe by solving (20). Then, by using (19) the 
value of Pi can be calculated. Then, the maximum entropy can be 
calculated too. By similar approach, Table 2 can be achieved. 

If the system with limited server capacity has information of 
mean customer number and busy period 1 —po, the maximum 
entropy model will be 



( N 



S.t.< 



N 



(23) 



'^iPi = Ls 

i=0 

^Pi>0, i = 0,\,2,---,N 



So we get 



where 



--cd' 



i=l,2, 



(24) 
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(25) 



that is 



-^r^=l-Po,c =L. (26) 

The solution is the root of the following equation: 



\-(N+\)d'^ + Nd^ + ^ _ L, 
l-d-d^ + dN+^ ~ l-po 



(27) 



By equation (27) with known N, L,, and po, the value of d can 
be decided and then c can be decided too by equation (26): then 
we can get the steady state distribution {/>,}. Similar results can be 
derived: 



if 



if 



l-po 



2 ' 
N+l 



then fl? > 1 , 



then I 



:1, 



if: 



then d<l. 



l-po 
L, iV+1 

l-Po 2 

In Tables 3 and 4, we assume N = 6 and po equals 0. 1 and 0.01, 
respectively; then we calculate the system performance distribution 
by changing the value of Lj. In Table 3, by using (26) the value of 
c and d can be got. Then substituting c and d into (24) the value of 
Pi can be calculated and the corresponding entropy can be 
calculated too. In similar procedure, we can get Table 4. 

3. The chi-square goodness of fit test 

Without any assumptions on the distribution of inter-arrival 
times and server times, we deduced the performance distributions 
of the queuing system by the maximum entropy principle above. Is 
this method effective and feasible? We will test our method by the 

goodness of fit test to determine this. 

Taking model (18) as an example, if we know the distribution of 
inter-arrival times and server times follow the Poisson process, 
then we get a M / M /I/ N queuing system. Its performance 
distribution is 



1-P 
l-p" + l 

Pn =P0P" 



Po 



(28) 



and the mean number of customers in the system is 



N 
11 = 0 



(N+l)p 



N+l 



l-p l-p 



N+l 



(29) 



We can decide the value of and L, and get the value of p by 
solving (29), the theoretic distribution can be calculated by using 
(28). The results are shown in Table 5. 

On the other side, if we only know N and Ls of the queuing 
system, according to (18) we can arrive at the maximum entropy 
distribution as shown in Table 1. Comparing Table 5 with 
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Table 1, it can be found that the maximum entropy distribution is 
very close to the theoretic distribution; that is, the is almost 
equal to 0. Hence, the maximum entropy distribution is a good 
estimation. 

Conclusions 

Queuing system analysis is usually based on some assumptions 
about the distributions of inter-arrival times and server times. This 
study shows that there is no need to assume those distributions, 
and if a queuing system is looked at as a black box, the system 
performance can be estimated by the maximum entropy principle 
with some easily accessible macro-level indexes. In this paper, 
some common queuing system are studied including queuing 
system with unlimited server capacity and queuing system with 
limited server capacity. By utilizing the principle of maximum 
entropy, and with known information of some easily accessible 
macro-level indexes such as mean number of customers in the 



system Z,j, system capacity N and mean server utilization of of 
1 —po, we demonstrate that maximum entropy method is a 
feasible and effective approach to estimate the system performance 
distribution. 

However, our study focused on single server queuing systems. 
For further research, multi-server queuing systems should be taken 
into consideration. For multi-server queuing system, more factors, 
for example rjueuing rules, server layout, system capacity et.al., 
should be considered. Hence, it will be more complicated. 
However, with assuming on those factors and observed indexes 
as presented in this paper, we can that our methods will be 
applicable in those circumstances also. 
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